Nested Datatypes 



Richard Bird 1 and Lambert Meertens 2 

1 Programming Research Group, Oxford University 
Wolfson Building, Parks Road, Oxford, 0X1 3QD, UK 
birdQcomlab . ox . ac . uk 
2 CW1 and Department of Computer Science, Utrecht University 
P.O. Box 94079, 1090 GB Amsterdam, The Netherlands 
lambert@cwi . nl 



Abstract. A nested datatype, also known as a non-regular datatype, is 
a parametrised datatype whose declaration involves different instances of 
the accompanying type parameters. Nested datatypes have been mostly 
ignored in functional programming until recently, but they are turning 
out to be both theoretically important and useful in practice. The aim of 
this paper is to suggest a functorial semantics for such datatypes, with 
an associated calculational theory that mirrors and extends the standard 
theory for regular datatypes. Though elegant and generic, the proposed 
approach appears more limited than one would like, and some of the 
limitations are discussed. 

Hark, by the bird's song ye may learn the 
nest. 

TENNYSON The Marriage of Geraint 



1 Introduction 



Consider the following three datatype definitions, all of which are legal Haskell 
declarations: 

data List a = NHL \ ConsL(a, List a) 
data Nest a = NUN | ConsN (a, Nest (a, a)) 
data Bush a = NUB \ ConsB (a, Bush (Bush a)) 

The first type, List a, describes the familiar t}'pe of cons-lists. Elements of the 
second type Nest a are like cons-lists, but the lists are not homogeneous: each 
step down the list, entries are "squared" . For example, using brackets and com- 
mas instead of the constructors NUN and ConsN. one value of type Nest Int. 
is 

[7, (1,2), ((6, 7), (7, 4)), (((2, 5), (7,1)), ((3, 8), (9, 3)))] 

This nest has four entries which, taken together, contain fifteen integers. 

In the third type Bush a, at each step down the list, entries are ''bushed". 
For example, one value of type Bush hit is 



Johan Jeuring (Ed.): MPC'98, LNCS 1422, pp. 52^Q 1998. 
(c) Springer- Verlag Berlin Heidelberg 1998 



Nested Datatypes 53 



[4, 

[8,[5],[[3]]], 

[[7],[],[[[7]]]], 

[[[],[[<)]]]] 

] 

This bush contains four entries, the first of which is an element of Int, the 
second an element of Bush Int. the third an element of Bush (Bush Int), and so 
on. In general, the n-th entry (counting from 0) of a list of type Bush a has type 
Bush 71 a. 

The datatype List a is an example of a so-called regular datatype, while 
Nest a and Bush a are examples of non-regular datatypes. Mycroft ^3 calls 
such schemes polymorphic recursions. We prefer the term nested datatypes. In a 
regular datatype declaration, occurrences of the declared type on the right-hand 
side of the defining equation are restricted to copies of the left-hand side, so 
the recursion is "tail recursive 11 . In a nested datatype declaration, occurrences 
of the datatype on the right-hand side appear with different instances of the 
accompanying type parameter(s), so the recursion is "nested' 1 . 

In a language like Haskell or ML, with a Hindley-Milner type discipline, it is 
simply not possible to define all the useful functions one would like over a nested 
datatype, even though such datatype declarations are themselves perfectly legal. 
This remark applies even to recent extensions of such languages (in particular, 
Haskell 1.4), in which one is allowed to declare the types of problematic functions, 
and to use the type system for checking rather than inferring types. To be sure, a 
larger class of functions can now be defined, but one still cannot define important 
generic functions, such as fold, over nested t} r pes. 

On the other hand, the most recent versions of Hugs and GHC (the Glas- 
gow Haskell Compiler) both support so-called rank-2 type signatures, in which 
one can universally quantify over type constructors as well as types (see ^3). 
By using such signatures one can construct most of the functions over nested 
datatypes that one wants. We will return to this point below. However, rank-2 
type signatures are not yet part of standard Haskell. 

The upshot of the current situation is that nested datatypes have been rather 
neglected in functional programming. However, they are conceptually important 
and evidence is emerging (e.g. ^^^^3) of their usefulness in functional data 
structure design. A brief illustration of what they can offer is given in SectionQ 

Regular datatypes, on the other hand, are the bread and butter of func- 
tion;!] programming. Recent work on polytypic programming (e.g. ^^^3) nas 
systematised the mathematics of program construction with regular datatypes 
by focusing on a small number of generic operators, such as fold, that can be 
defined for all such types. The basic idea, reviewed below, is to define a regular 
datatype as an initial object in a category of F-algebras for an appropriate func- 
tor F. Indeed, this idea appeared much earlier in the categorical literature, for 
instance in ^fl. As a consequence, polytypic programs are parametrised by one 
or more regular functors. Different instances of these functors yield the concrete 
programs we know and love. 
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The main aim of this paper is to investigate what form an appropriate func- 
torial semantics for nested datatypes might take, thereby putting more 'poly' 
into 'polytypic'. The most appealing idea is to replace first-order functors with 
higher-order functors over functor categories. In part, the calculational theory 
remains much the same. However, there are limitations with this approach, in 
that some expressive power seems to be lost, and some care is needed in order 
that the standard functorial semantics of regular datatypes may be recovered as 
a special case. It is important to note that we will not consider datatype dec- 
larations containing function spaces in this paper; see for ways of dealing 
with function spaces in datatype declarations. 

2 An example 

Let us begin with a small example to show the potential of nested datatypes. 
The example was suggested to us b}' Oege de Moor. In the De Bruijn notation 
for lambda expressions, bound variables introduced by lambda abstractions are 
represented by natural numbers. An occurrence of a number n in an expression 
represents the bound variable introduced by the n-th nested lambda abstraction. 
For example, 0(1,1) represents the lambda term 

Xx.Xy.x(yy) 

On the other hand, 0 (w 1) represents the lambda term 

Xx.Xy.x (w y) 

in which w is a free variable. 

One way to capture this scheme is to use a nested datatype: 

data Term a = Vara \ App (Term a, Term a) \ Abs (Term (Bind a)) 
data Bind a = Zero \ Succ a 

Elements of Term a are either free variables (of type Vara), applications, or 
abstractions. In an abstraction, the outermost bound variable is represented by 
Var Zero, the next b}' Var (Succ Zero), and so on. Free variables in an abstraction 
containing n nested bindings have type Var (Succ 11 a). The type Term a is nested 
because Bind a appears as a parameter of Term on the right-hand side of the 
declaration. 

For example, A x.Xy.x (w y) may be represented by the following term of type 
Term Char: 

Abs (Abs (App ( Var Zero, App ( Var (Succ (Succ V)), Var (Succ Zero))))) 

The closed lambda terms - those containing no free variables - are elements of 
Term Empty, where Empty is the empty type containing no members. 

The function abstract, which takes a term and a variable and abstracts over 
that variable, can be defined in the following way: 

abstract :: (Term a, a) —* Term a 

abstract (t, x) = Abs (lift (t, x)) 
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The function lift is defined by 

lift :: (Term a, a) —* Term (Bind a) 

lift ( Var y,x) — if x = y then Var Zero else Var (Succ y) 

lift (App (u, v), x) = App (lift (u, x), lift (v, x)) 

lift (Abs t, x) = Abs (lift, (t, Succx)) 

The /^-reduction of a term is implemented by 

reduce :: (Term a. Term a) —* Term a 

reduce (Abs s, = subst (s,t) 

where 

subst :: (Term (Bind a) , Term a) — > Term a 

subst ( Var Zero, t) = t 
subst ( Var (Succ x) } t) = Var x 

subst (App (u. w), t) = App (subst (u, t), subst (v, t)) 
subst (Abs S, t) = Abs (subst (s, term Succ t)) 

The function termf maps / over a term: 

term :: (a — * b) — * (Term a —* Term b) 

termf (Var x) = Var (fx) 
termf (App (u, v)) = App (termf u, termf v) 
termf (Abs t) = Abs (term (bind f) t) 

The subsidiary function bind f maps / over elements of Bind a: 

bind :: (a — + b) — + (Bind a — ► Bind b) 

bind f Zero = Zero 
bindf (Succ x) = Succ (f x) 

It is a routine induction to show that 

reduce (abstract (t,x). Var x) = t 

for all terms t of type Term a and all x of type a. 

Modulo the requirement that a and Bind a be declared as equality types 
(because elements are compared for equality in the definition of lift) the programs 
above are acceptable to Haskell 1.4. provided the type signatures are included 
as part of the definitions. 



3 Datatypes as initial algebras 

The standard semantics (see e.g. ^^j) of inductive datatypes parametrised by 
//. type parameters employs functors of type C x • ■ - x C — * C. where the product 
has 7i + l occurrences of C. For simplicity, we will consider only the case n = 1. 
The category C cannot be arbitrary: essentiall}', it has to contain finite sums and 
products, and colimits of all ascending chains. The category Fun (also known 
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as Set), whose objects are sets and whose arrows are typed total functions, has 
everything needed to make the theory work. 

To illustrate, the declaration of List as a datatype is associated with a binary 
functor F whose action on objects of C x C is defined by 

F(a, b) = l + axb 

Introducing the unary functor F a , where F a (b) = F(a, 6), the declaration of 
List a can now be rewritten in the form 

data List a F a (List a) 

in which a a :: F a (List a) —* List a. For the particular functor F associated 
with List, the arrow a a takes the form (NilL a , ConsL a ), where NilL a :: 1 — * 
List a and ConsL„ :: ax List a —* List a. This declaration can can be 
interpreted as the assertion that the arrow a a and the object List a are the 
"least" values with this typing. More precisely, given any arrow 

f::F a (b)^b 

the assertion is that there is a unique arrow h : : List a —* b satisfying the 
equation 

h-a a =f-F(id a ,h) 

The unique arrow h is denoted by fold} . The arrow h is also called a catamor- 
phism, and the notation ([/]) is also used for foldf. In algebraic terms, List a is 
the carrier of the initial algebra a a of the functor F a and foldf is the unique 
F„-homomorphism from the initial algebra to /. 

A surprising number of consequences flow from this characterisation. In par- 
ticular, fold a a is the identity arrow on List a. Also, one can show that a a is an 
isomorphism, with inverse fold (F(id a ,a a )). As a result, one can interpret the 
declaration of List as the assertion that, up to isomorphism, List a is the least 
fixed point of the equation x = F(a, x). 

The type constructor List can itself can be made into a functor by defining 
its action on an arrow / : a — * b by 

list f = fold (a b -F(fJd)) 

In functional programming listf is written map f '. Expanding the definition of 
fold, we have 

listf • a a = <*b • F(f, list f) 

This equation states that a is a natural transformation of type a :: G — * List, 
where G a = F(a, List a). 

The most important consequence of the characterisation is that it allows one 
to introduce new functions by structural recursion over a datatype. As a simple 
example, fold (zero, plus) sums the elements of a list of numbers. 

Functors built from constant functors, type functors (like List), the identity 
and projection functors, using coproduct, product, and composition operations, 
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are called regular functors. For further details of the approach, consult, e.g., 
or 1 . 

For Nest and Bush the theory above breaks down. For example, introducing 
Q a = a x a for the squaring functor, the corresponding functorial declaration 
for Nest would be 

data Nest a <2i. F(a, Nest (Q a)) 

where F is as before, and a a applies NUN to left, components and ConsN to 
right components. However, it is not clear over what class of algebras a a can be 
asserted to be initial. 



4 A higher-order semantics 

There is an appealing semantics for dealing with datatypes such as Nest and 
Bush, which, however, has certain limitations. We will give the scheme, then 
point out the limitations, and then give an alternative scheme that overcomes 
some of them. 

The idea is to use higher-order functors of type 

Nat(C) — Nat(C) } 

where Nat(C) is the category whose objects are functors of type C — * C 
and whose arrows are natural transformations. We will use calligraphic letters 
for higher-order functors, and small Greek letters for natural transformations. 
Again, the category C cannot be arbitrary, but taking C = Fun gives everything 
one needs. Here are three examples. 

Example 1. The declaration of List can be associated with a higher-order functor 
T defined on objects (functors) by 

F(F)(a) = l + axF(a) 
F(F)(f) = id l+ fx F(f) 

These equations define f(F) to be a functor for each functor F. The functor T 
can be expressed more briefly in the form 

?{F) = K\ + IdxF 

The constant functor K a delivers the object a for all objects and the arrow 
id a for all arrows, and Id denotes the identity functor. The coproduct (+) and 
product (x) operations are applied point wise. 

The action of T on arrows (natural transformations) is defined in a similar 
style by 

F(rj) = idm + id x // 

Here, id^x delivers the identity arrow id\ for each object of C. If rj :: F — * G, 
then Tift) :: F{F) -» F{G). We have ?{id) = id, and ■ = ■ ^(^), 
so T is itself a functor. 
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The previous declaration of List can now be written in the form 
data List T{List) 
and interpreted as the assertion that a is the initial ^"-algebra. 

Example 2. The declaration of Nest is associated with a functor defined on 
objects (functors) by 

F(F)(a) = l + axF{Qa) 
T(F)(f) = id l +fxF(Qf) 

where Q is the squaring functor. More briefly, 

T{F) = Kl + Idx(F-Q) 

where F • Q denotes the (functor) composition of F and Q. Where convenient, 
we will also write this composition as FQ for brevity. 

The action of T on arrows (natural transformations) is defined by 

T{r)) = id K \ + idxrjQ 

where rjQ :: FQ — GQ if rj :: F —> G. 

Example 3. The declaration of Bush is associated with a functor !F, defined on 
functors by 

F{F) =Kl + Idx(F-F) 

and on natural transformations by 

F(rj) = idxi + id x (rf* rj) 

The operator * denotes the horizontal composition of two natural transforma- 
tions. If 9 :: F —> G and ip :\ H —> N , then 9 *ip :\ FH — ► GN is denned by 
9-kij) = ON • Ftl>. In particular, if rj :: F —> G, then r)*rj::FF-> GG. 

Consider again the declaration of Nest given in the Introduction, and rewrite 
it in the form 

data Nest *2- T(Nest) 

The assertion that a is the initial ^"-algebra means that for any arrow <p :: 
F(F) —* .F, there is a unique arrow 9 :: Nest —* F satisfying the equation 

The unique arrow 9 is again denoted by fold (p. 

We can express the equation above in Haskell. Note that for the particular 
functor T associated with Nest % the arrow <p takes the form <p = (e, where 
£ :: Kl — ► F and tp :: Id x FQ — * F. For any type a, the component e a is 
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an arrow delivering a constant e of type F a, while ip a is an arrow / of type 
(a, F(a, a)) —* F(a). Hence we can write 

fold (e J) NUN = e 

fold (e.f) (ConsN (x, xps)) = f(xJold (ej) xps) 

However, no principal type can be inferred for fold under the Hindley-Milner type 
discipline, so the use of fold in programs is denied us. Moreover, it is not possible 
to express the type of fold in an}' form that is acceptable to a standard Haskell 
type checker. On the other hand, in GHC (The Glasgow Haskell Compiler) one 
can declare the type of fold by using a rank-2 type signature: 

fold :: (V/.V6. ((Va.f a), (Va.(aJ(a, a)) — / a)) — Nestb^ f b) 

This declaration uses both local universal quantification and abstraction over a 
type constructor. Such a signature is called a rank-2 type signature. With this 
asserted type, the function fold passes the GHC type-checker. 

Observe that in the proposed functorial scheme, unlike the previous one for 
regular datatypes, the operator fold takes natural transformations to natural 
transformations. In particular, the fact that Nest is a functor is part of the 
assertion that Nest is the least fixed point of f r . The arrow nestf cannot be 
defined as an instance of fold since it is not a natural transformation of the right 
type. 

The typing a :: f(Nest) — ► Nest means that, given / :: a — ► 6, the following 
equation holds: 

nestf ■ a a = at • T(nest) f 

We can express this equation at the point level by 

nestf NUN = NUN 

nest f (ConsN (x, xps)) = ConsN (fx, nest (square f) xps) 

where square f (x, y) = (f x.f y) is the action on arrows of the functor Q. The 
fact that nest is uniquely defined by these equations is therefore a consequence 
of the assertion that a is a natural transformation. 

Exactly the same characterisation works for Bush. In particular, the arrow 
bushf satisfies 

bush f NUB = NUB 

bush f (ConsB (x, xbs)) = ConsB (f x, bush (bushf) xbs) 
5 Examples 

To illustrate the use of folds over Nest and Bush, define r :: Q —* List by 
r(x,y) = [x,y] 

Using r and the natural transformation coneat :: List • List — > List, we have 
eoncat • listr :: List ■ Q — > List, and so 

&List • T(eoncat • listr) :: T(List) —> List 
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where T(F) = Kl + Id x FQ is the higher-order functor associated with Nest. 
The function listify, defined by 

listify = fold (aiist * T(coneat • list r)) 
therefore has type listify :: Nest — ► List. For example, listify takes 

[0,(1,1), ((2, 2), (3, 3))] to [0,1,1,2,2,3,3] 
The converse function nestify :: List —* Nest can be defined by 

nestify = fold (a Nest * J 7 (nest 5)) 

where f(F) = Kl + Id x F is the higher-order functor associated with List, and 

5 a = (a, a) has type S :: Id — ► Q. For example, nestify takes 

[0,1,2] to [0,(1,1), ((2, 2), (2, 2))] 
For another example, define a :: Q —> Bush by 
= [ar,[j/]] 

Then bush a :: • Q — * Bush • Bush, and so 

<*Bush ' T {hush a) :: J 7 (Bush) —* Bush 
where P(F) = Kl + Id x FQ is the functor associated with Nest. Hence 

bushify = fold (a Dusk ■ T '(bush a)) 
has type bushify :: Nest —* Bush. For example, bushify sends 

[1, (2, 3), ((4, 5), (6, 7))] to [1, [2, [3]], [[4, [5]], [[6, [7]]]]] 

6 The problem 

The basic problem with the higher-order approach described above concerns 
expressive power. Part of the problem is that it does not generalise the standard 
semantics for regular datatypes; in particular, it does not enable us to make use 
of the standard instances of fold over such datatypes. To see why not, let us 
compare the two semantics for the datatype List. 

Under the standard semantics, foldf :: List a —* b when f :: 1 + a x b —* b. 
For example, 



fold (zero , plus) :: Listlnt —* Int 

sums a list of integers, where zero :: 1 — * 
plus :: Int x Int —* Int is binary addition. 
As another example, 

fold (nil, eat) :: List (List a) —* List a 



Int is a constant delivering 0, and 
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concatenates a list of lists; this function was called concat above. The binary 
operator cat has type cat :: List a x List a — ► List a and concatenates two lists. 

Under the new semantics, foldip :: List —* F when <p :: Kl + Id x F —* F. 
We can no longer sum a list of integers with such a fold because plus is not a 
natural transformation of the right type. For fold {zero, plus) to be well- typed 
we require that plus has type plus :: Id x KInt — ► if/rtf. Thus, 

p/us a :: a x Jni —* Int 

for all a, and so plus would have to ignore its first argument. 

Even worse, we cannot define concat :: List • List — » List as an instance 
of fold, even though it is a natural transformation. The binary concatenation 
operator cat does not have type 

cat :: Id x List — » List 

because again it would have to ignore its first argument. Hence fold (nil, cat) is 
not well-typed. 

On the other hand. et^est ■ T(nest S) does have type Kl + Id x JVesi — * Atestf, 
so the definition of nestify given in the previous section is legitimate. 

Putting the problem another way, in the standard semantics, foldf is defined 
by providing an arrow / :: F(a, b) — * b for a fixed a and 6; we cannot in general 
elevate / to a natural transformation that is parametric in a. 

7 An alternative 

Fortunately, for lists and other regular datatypes, there is a way out of this 
particular difficulty. Using the isomorphism defining List, the functor List • F 
satisfies the isomorphism 

List • F (Kl + Id x List) ■ F = Kl + F x (List ■ F) 

Hence List • F is isomorphic to the "higher-order" datatype Listr F, declared by 

data Listr F Kl + F x Listr F 

We can write the functor on the right as !F(F , Listr F), where T now is a higher- 
order binaiy functor of type 

Nat(C) x Nat(C) -> Nat(C) 

Over the higher-order datatype Listr F, the natural transformation fold <p takes 
an arrow ip :: K1 + F x G —* G, and has type fold tp :: Listr F — ► G. If we change 
Listr F to List ■ F in this signature, we have a useful fold operator for lists. In 
particular, 

fold (zero, plus) :: List • KInt — > KInt 

since (zero, plus) :: Kl + KInt x KInt — * KInt. The arrow fold (zero, plus) of 
Nat(C) is a natural transformation; since List • KInt = K (List Int). its compo- 
nent for any a is the standard fold fold (zero, plus) :: List Int — ► /n£. 
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By a similar device, all folds in the standard semantics are definable as folds 
in the new semantics, simply by lifting the associated algebra to be a natural 
transformation between constant functors. 

More precisely, define Type a to be the least fixed point of a regular func- 
tor F a , where F a (b) = F(a : b). Furthermore, define Typer G to be the least 
fixed point of T G , where T G {K) = T(G, H) and F(G, H)x = F(Gx } Hx) for all 
objects x. Take an algebra / :: F(a,b) —* b. and construct the natural transfor- 
mation :: T(Ka, Kb) —* Kb by setting ip = Kf. This is type correct since 

T(Ka.Kb)x = F(Ka(x),Kb(x)) = F(a. r b) and Kb(x) = b 

Then fold/ :: Type a — ► 6, and fold <p :: Typer Ka — * Kb satisfy 

fold<p = K(foldf) 

under the isomorphism Typer Ka = K(Type a). 

Thus, not only do we generalise from the defining expression for List by 
replacing occurrences of List by G, we also generalise by replacing occurrences 
of Id by a functor F. 

However, the same idea does not work for nested datatypes such as Nest. 
This time we have 

Nest • F ^ (Kl + Id x (Nest • Q)) • F = Kl + F x (Nest - Q • F) 

The type Nest • F is quite different from the datatype defined by 

data NestrF Kl + F x ((Nestr F) ■ Q) 

For example, Nest (List a) is the type of nests of lists over a, so the n-th entiy 
of such a nest has type Q n (List a). On the other hand the n-th entry of a nest 
of type Nestr List a has type List (Q n a). 

Even more dramatically, the type Nest Int gives a nest of integers, but 
Nestr KInt b is isomorphic to ordinary lists of integers for all b. More gener- 
ally, Nestr K a is the constant functor K(List a). 

On the other hand, we have Nest = Nestr Id, so the higher-order view is 
indeed a generalisation of the previous one. 

8 Reductions 

Replacing higher-order unary functors by higher-order binary functors enables us 
to integrate the standard theory of regular datatypes into the proposed scheme. 
Unfortunately, while the higher-order approach is elegant and generic, it seems 
limited in the scope of its applicability to nested datatypes, which is restricted 
to folding with natural transformations. For example, one cannot sum a nest of 
integers with a fold over nests. Such a computation is an instance of a useful 
general pattern called a reduction. It is possible to define reductions completely 
generically for all regular types (see ^3), but we do not know at present whether 
the same can be done for nested datatypes. 
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One way to sum a nest of integers is by first listifying the nest and then 
summing the result with a fold over lists. More generally, this strategy can be 
used to reduce a nest with an arbitrary binary operator 0 and a seed e. For 
example, 

[xq, (xi t X2), ((a&,a*),(afc,afc))] 

reduces to 

^e^e^e-'-e^e e))) 

It can be argued that this strategy for reducing over nests is unsatisfactory 
because the structure of the nest entries is not reflected in the way in which 0 
is applied. Better is to introduce a second operator 0 and reduce the nest above 
to 

xq 0 ((xi 0 3^)0 (((x 3 0 xi) 0 (a* 0 a*)) 0 e)) 

By taking 0 to be 0, we obtain another way of reducing a nest. 

The above pattern of computation can be factored as a fold over lists after a 
reduction to a list: 

fold (e, 0) ■ reduce (0) 

With (0) :: Qa — » a, the function reduce (0) has type Nest a — ► List a. For 
example, applied to the nest above, reduce (0) produces 

[a&, Xi 0 22, (x 3 0 x 4 ) 0 (x 5 0 2fe)] 

There is no problem with defining reduce. In a functional st}'le we can define 

reduce op NUN = NHL 

reduce op (ConsN xps)) = ConsL (x, reduce op (nest op xps)) 

In effect, reduce op applies the following sequence of functions to the correspond- 
ing entries of a nest: 

[id, op, op • square op, op • square op • square (square op), . . .] 

The n-th element of this sequence has type Q n a —* a when op :: Q a — * a. 
The reduction of a bush proceeds differently: 

reduce (e, op) NUB = e 

reduce (e, op) (ConsB (x, xbs)) = 

op (x, reduce (e, op) (bush (reduce (e, op)) xbs)) 

At present we see no systematic wa}' of unifying reductions over nested datatypes, 
nor of relating them to the folds of previous sections. 
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9 Another approach 

There is a way that higher-order folds and the reductions of the previous section 
can he unified, but whether or not the method is desirable from a calculational 
point of view remains to be seen. It requires a different and more complicated 
notion of folding over a nested datatpe, one that involves an infinite sequence of 
appropriate algebras to replace the infinite sequence of differently typed instances 
of the constructors of the datatj'pe. We will briefly sketch the construction for 
the type Nest a. 

The basic idea is to provide an infinite sequence of algebras to replace the 
constructor a = (NUN, ConsN) of Nest, one for each instance 

a :: F(Q n a, Nest(Q n+1 a)) - Nest (Q n a) 

where n is a natural number and F(a, b) = 1 + a x b. For regular datatypes the 
application of foldf to a term can be viewed as the systematic replacement of 
the constructors by corresponding components of/, followed by an evaluation of 
the result. The same idea is adopted here for nested datatypes. However, whereas 
for regular datatypes each occurrence of a constructor in a term has the same 
typing, the same is not true for nested datatypes, hence the need to provide a 
collection of replacements. 

In more detail, consider the datatype NestAlgs defined by 

data NestAlgs G (a, b) = Cons (F(a, G{Qb)) -> Gb, NestAlgs G (Qa, Qb)) 

The datatype NestAlgs is a coinductive, infinite, nested datatype. The n-th entry 
of a value of type NestAlgs G(a,b) is an algebra of type 

F(Q n a, G(Q 1l+l b))->G(Q n b) 
Now for fs :: NestAlgs G (a, 6), define foldfs :: Nest a — * Gb by the equation 

foldfs ■ a = head fs ■ F (id, fold (tailfs)) 

where 

head (Cons (fjs)) =/ 
tail (Cons {f Js)) = fs 

Equivalently, 

fold(Cons(f,fs))-a = f-F(id, foldfs) 

To illustrate this style of fold, suppose / :: a —> b and define generate f :: 
NestAlgs Nest (a, b) by 

generate f = Cons (a • F(f, id), generate (square f)) 

Then fold (generate f) :: Nest a —* Nest b, and in fact 

nest f = fold (generate f) 
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The functorial action of Nest on arrows can therefore be recovered as a fold. The 
proof of nest (f ■ g) = nestf • nest g makes use of eoinduction. 

As another example, suppose <p :: T(Id, GQ) — * G is a natural transforma- 
tion, where T(M, N)a = F(Ma, Na). Define repeat^ :: NestAlgs G by 

repeat ip = Cons (<p, repeat <pQ) 

For each type a we have (repeat ip) a :: NestAlgs G (a, a). The relationship be- 
tween the higher-order folds of the previous sections and the current style of 
folds is that 

fold ip = fold ( repeat <p) 

In particular, fold (repeat a) = id :: Nest —* Nest. 

We can also define reductions as an instance of the new folds. Suppose 
/ :: F(a, a) — * a, so / = (/o,/i), where f\ :: Qa — * a. Define redalgsf :: 
NestAlgs Ka (a, b) by 

redalgs f = red id 

where red k = Cons (f • F(k, id), red (fi ■ square k)) 

We have fold (redalgsf) :: Nest a — * a, and we claim that 

reduce f = fold (redalgsf) 



10 Conclusions 



The results of this investigation into nested datatypes are still incomplete and 
in several aspects unsatisfactory. The higher-order folds are attractive, and the 
corresponding calculational theory is familiar, but they seem to lack sufficient 
expressive power. The approach sketched in the previous section for Nest is more 
general, but brings in more machinery. Furthermore, it is not clear what the right 
extension is to other nested datatypes such as Bush. 

We have also ignored one crucial question in the foregoing discussion, namely, 
what is the guarantee that functors such as Nest and Nestr do in fact exist 
as least fixed points of their defining equations? The categorical incantation 
ensuring the existence of an initial F-algebra in a co-complete category C is 
that, provided F is co- continuous, it is the colimit of the chain 

0 ^ FO «-* FFO ^ ■ ■ • 

The category Fun has everything needed to make this incantation work: Fun 
is co-complete (in fact, bi-complete) and all regular functors F on Fun are co- 
continuous. The proof for polynomial functors can be foimd in and the 
extension to type functors is in 

Moreover, the category Nat (F\in) inherits co-completeness from the base 
ategory Fun (see ^^Q). We believe that all regular higher-order functors are 
co-continuous, though we have not yet found a proof of this in the literature, so 
the existence of datatypes like Nest and Bush is not likely to be problematic. 
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If we adopt the higher-order approach, then there is a need to give a system- 
atic account of reductions over a nested datatype. If the alternative method of 
the previous section proves more useful, then there is a need to give a systematic 
account of the method, not only for an arbitrary inductive nested datatype, but 
also for coinductive nested datatypes. 

Finally, in Q (see also Q) the idea was proposed that a datatype was a 
certain kind of functor called a relator, together with a membership relation. 
It needs to be seen how the notion of membership can be extended to nested 
datatypes 
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